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DETAILED ACTION 



Claim Rejections - 35 USC §102 



1. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 



A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 35 1(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 



2. Claims 1 and 10-13 are rejected under 35 U.S.C. 102(b) as being anticipated by U.S. 
Patent No. 5,893,095 issued to Jain et al. (hereafter Jain '095). 
Claim 1: 



Regarding Claim 1, Jain '095 discloses a content based image query engine. Specifically, 
Jain c 095 discloses: a method of seeking images, from an example image, from amongst a 
plurality of images stored in a database, each of the stored images being associated with a data 
item of a first type, referred to as an index of the stored image, representing at least one 
characteristic of the visual content of the image (Jain '095: Abstract; col. 3, In. 58 to col. 4, In. 
4), said method comprising the following steps: 

- receiving a data item of a second type representing the location of at least one region of 

interest in the example image (Jain '095: col. 4, Ins. 21-28; col. 35, Ins. 17-18); 
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- for each region of interest, ROI r , receiving a data item of a third type, V r , indicative of a 
type of taking into account of the content of the said region of interest for the seeking of 
images (Jain '095: col. 8, Ins. 6-15); 

- calculating a data item of a fourth type, referred to as the index of the example image, 
representing at least one characteristic of the visual content of the example image, the 
method of calculating said data item of the fourth type depending on said data item of the 
second type and said data items of the third type (Jain '095: col. 8, Ins. 17-20, note that a 
primitive may be a composite function); 

- calculating a similarity between the example image and each of the images amongst at 
least one subset of the stored images, said similarity being calculated from said data item 
of the first type associated with the stored image and the data item of the fourth type 
associated with the example image (Jain '095: col. 8, Ins. 20-35; col. 7, Ins. 9-23); and 

- supplying at least one image, referred to as the result image, in the database, said at least 
one result image being selected from amongst said stored images in the database 
according to its degree of similarity with said example image (Jain '095: col. 9, In. 64 to 
col. 10. In. 10). 

Claim 3: 

Regarding Claim 3, Jain '095 discloses all the limitations of Claim 1 (supra). Further 
note that Jain £ 095 additionally discloses: wherein said data item of the first type, called index of 
the stored image, associated with each of said stored images, consists of a histogram, of colors 
relating to the global content of the image (Jain '095: col. 7, Ins. 50-53). 
Claim 10: 
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Regarding Claim 10, Jain '095 discloses all the limitations of Claims 1 (supra). Further 
note that Jain '095 additionally discloses: said data item of the second type representing the 
location of at least one region of interest in the example image consists of a set of two- 
dimensional points indicative of the shape of said at least one region of interest and its location 
in the image plane of said example image (Jain '095: col. 35, Ins. 17-18 - note that an image is 
two-dimensional and specifying a "portion of the image" as per Jain '095 includes indicating the 
shape and location of a region of interest). 
Claim 11: 

Regarding Claim 11, Jain '095 discloses the method of Claim 1 (supra). Further note that 
Jain '095 additionally discloses the device embodiment of said methods (Jain '095: col. 10, Ins. 
11-40). 
Claim 12: 

Regarding Claim 12, Jain '095 discloses the method of Claim 1 (supra). Further note that 
Jain '095 additionally discloses a computer comprising means embodiment of said methods (Jain 
'095: col. 10,1ns. 11-40). 
Claim 13: 

Regarding Claim 13, Jain '095 discloses disclose all the limitations of Claim 1 1 (supra). 
Further note that Jain '095 additionally discloses the computer embodiment of Claim 1 1 (Jain 
'095: col. 10,1ns. 11-40). 

3. Claims 1,3, and 10-13 are rejected under 35 U.S. C 102(e) as being anticipated by U.S. 
Patent No. 6,373,979 issued to Wang (hereafter Wang '979). 
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Claim 1: 

Regarding Claim 1, Wang '979 discloses a content based image query engine. 
Specifically, Wang '979 discloses: a method of seeking images, from an example image, from 
amongst a plurality of images stored in a database, each of the stored images being associated 
with a data item of a first type, referred to as an index of the stored image, representing at least 
one characteristic of the visual content of the image (Wang '979: Abstract, Fig. 5, Fig. 8A, col. 
7, Ins. 10-12), said method comprising the following steps: 

- receiving a data item of a second type representing the location of at least one region of 
interest in the example image (Wang '979: col. 4, Ins. 27-32); 

- for each region of interest, ROI r , receiving a data item of a third type, V r , indicative of a 
type of taking into account of the content of the said region of interest for the seeking of 
images (Wang '979: col. 4, Ins. 44-46); 

- calculating a data item of a fourth type, referred to as the index of the example image, 
representing at least one characteristic of the visual content of the example image, the 
method of calculating said data item of the fourth type depending on said data item of the 
second type and said data items of the third type (Wang '979: col. 5, Ins. 6-20, note the 
weighted similarity function); 

- calculating a similarity between the example image and each of the images amongst at 
least one subset of the stored images, said similarity being calculated from said data item 
of the first type associated with the stored image and the data item of the fourth type 
associated with the example image (Wang '979: col. 5, Ins. 22-30); and 
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- supplying at least one image, referred to as the result image, in the database, said at least 
one result image being selected from amongst said stored images in the database 
according to its degree of similarity with said example image (Wang '979: Fig. 8A, col. 
7, Ins. 21-23). 
Claim 3: 

Regarding Claim 3, Wang '979 discloses all the limitations of Claim 1 (supra). Further 
note that Wang '979 additionally discloses: wherein said data item of the first type, called index 
of the stored image, associated with each of said stored images, consists of a histogram, of colors 
relating to the global content of the image (Wang '979: col. 4, Ins. 44-45). 
Claim 10: 

Regarding Claim 10, Wang c 979 discloses all the limitations of Claims 1 (supra). Further 
note that Wang c 979 additionally discloses: said data item of the second type representing the 
location of at least one region of interest in the example image consists of a set of two- 
dimensional points indicative of the shape of said at least one region of interest and its location 
in the image plane of said example image (Wang '979: col. 4, Ins. 27-36 - note that an image is 
two-dimensional and specifying the "attributes of the segments" as per Wang '979 reads on the 
shape and location of a region of interest). 
Claim 11: 

Regarding Claim 1 1, Wang '979 discloses the method of Claims 1 and 2 (supra). Further 
note that Wang £ 979 additionally discloses the device embodiment of said methods (Wang '979: 
col. 9, Ins. 4-10). 
Claim 12: 
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Regarding Claim 12, Wang '979 discloses the method of Claims 1 and 2 (supra). Further 
note that Wang '979 additionally discloses a computer comprising means embodiment of said 
methods (Wang '979: col. 9, Ins. 4-10). 
Claim 13: 

Regarding Claim 13, Wang '979 discloses disclose all the limitations of Claim 1 1 
(supra). Further note that Wang c 979 additionally discloses the computer embodiment of Claim 
11 (Wang £ 979: col. 9, Ins. 4-10). 



Claim Rejections - 35 USC §103 
4. The following is a quotation of 35 U.S.C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



5. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Wang '979 in view 
of U.S. Patent No. 6,230,154 issued to Raz et al. (hereafter Raz '154). 
Claim 2: 

Regarding Claim 2, Wang '979 discloses all the limitations of Claim 1 (supra). Further 
note that Wang '979 additionally discloses: wherein said data item of the third type V r , 
associated with a region of interest ROI r , is a scalar data item which can take all the values lying 
between a predefined lower value Vmin, and a predefined higher value Vmax, (Wang '979: col. 5, 
Ins. 22-39) and wherein: 
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- if said data item of the third type V, is equal to the predefined lower value Vmin, the 
content of the images sought must not be similar to the content of the corresponding 
region of interest ROI r (Wang £ 979: col. 5, Ins. 29-33); and 

- if said data item of the third type V r , is equal to the predefined higher value Vmax, the 
content of the images sought must be similar to the content of the corresponding region 
of interest ROI r (Wang '979: col. 5, Ins. 27-31). 

However, Wang '979 does not explicitly disclose: 

- if said data item of the third type V r , lies strictly between the lower predefined value Vmin 
and the higher predefined value Vmax, the content of the images sought must be more or 
less similar to that of the corresponding region of interest ROI r depending on whether the 
value of V r , is close to Vmax or is close to Vmin, the overall content of the example image 
also having to be taken into consideration. 

Raz ' 154 discloses a hyperbox that defines whether a retrieved item is similar. 
Specifically, Raz '154 discloses: 

- if said data item of the third type V r , lies strictly between the lower predefined value Vmi n 
and the higher predefined value Vmax, the content of the images sought must be more or 
less similar to that of the corresponding region of interest ROI r depending on whether the 
value of V r , is close to Vmax or is close to Vmin, the overall content of the example image 
also having to be taken into consideration (Raz c 154: Ins. 14-27). 

It would have been obvious to a person having ordinary skill in the art to augment the 
similarity criteria of Wang '979 with the hyperbox of Raz 4 154. The motivation to accomplish 
said augmentation is suggested by Raz '154 which discloses that applying the hyperbox of Raz 
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' 154 provides a particularly advantageous way to view and conceptualize similarity relationships 
between database items (Raz £ 154: col. 3, Ins. 23-38). 

6. Claims 3-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over Wang £ 979 
and Raz '154, in view of Jain '095. 
Claim 4: 

Regarding Claim 4, Wang '979 and Raz ' 154 in combination disclose all the limitations 
of Claim 3 (supra). Further note that Wang '979 and Raz ' 154 in combination additionally 
disclose that a condition can be set for: 

- if all said data items of the third type are equal to said lower predefined value Vmin, or if 
all said data of the third type are equal to said higher predefined value Vmax; or 

- if each of said data of the third type is equal to Vmm or equal to Vmax (Raz ' 1 54: Ins. 14-27 
- note the hyperbox of Raz '154 reads on this range setting) 

However, Wang '979 and Raz ' 154 in combination do not explicitly disclose that the action 
taken on the above condition is: 

- said step of calculating a data item of a fourth type, called index of the example image, 
includes a step of calculating a vector, (Gr(Q)), each component of which consists of the 
histogram of colors representing the visual content of one of said regions of interest, said 
vector constituting the index of said example image. 

Jain '095 discloses: 

- said step of calculating a data item of a fourth type, called index of the example image, 
includes a step of calculating a vector, (Gr(Q)), each component of which consists of the 
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histogram of colors representing the visual content of one of said regions of interest, said 
vector constituting the index of said example image (Jain '095: col. 8, Ins. 32-34 - note 
that the composite metric of Jain '095 reads on the index of Applicant). 
It would have been obvious to a person having ordinary skill in the art to set the 
composite metric of Jain '095 for the condition of Wang '979 and Raz * 154 in combination. The 
motivation to accomplish said setting is suggested by Jain '095 which discloses the necessity of 
combining into a composite metric different distance metric components of multiple primitives 
(i.e. computed statistical features of an image) (Jain '095: col. 8, Ins. 29-34). 
Claim 5: 

Regarding Claim 5, Wang '979, Raz '154, and Jain '095 in combination disclose all the 
limitations of Claim 4 (supra). Further note that Wang '979, Raz '154, and Jain '095 in 
combination additionally discloses: wherein if all said data of the third type are strictly between 
said lower predefined value V m i n and said higher predefined value Vmax, then said step of 
calculating a data item of a fourth type, called index of the example image (Raz ' 1 54: Ins. 14-27 
and Jain '095: col. 8, Ins. 29-34), includes the following steps: 

- calculating a matrix (W) with M rows and M columns, where M is a integer number 
designating the number of colors available, each element of whose diagonal corresponds 
to one of the M colors available, each of the elements of the diagonal having a value 
which is calculated as a function of the dominant character of the color associated with 
said element in said at least one region of interest associated with said example image, 
and of said data item of the third type associated with said at least one region of interest 
(Wang '979: col. 4, In. 62 to col. 5, In. 20 - note the "weighting matrix" of Wang '979); 



# 4l 
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- calculating the histogram of colors (H M (Q)) representing the overall visual content of said 
example image (Q) (Wang '979: col. 4, In. 62 to col 5, In. 20 - note the multiplication of 
the "weighting matrix" with the "histograms representing the color characteristics" of 
Wang '979); 

- defining said index of the example image (Q) as being the result (X(Q)) of the product of 
said matrix (W) and said histogram of colors (HmQ)) representing the overall visual 
content of said example image (Q) (Wang '979: col. 4, In. 62 to col. 5, In. 20 - note the 
use of the multiplication of the "weighting matrix" with the "histograms representing the 
color characteristics" for a "weighted similarity function" of Wang '979). 

Claim 6: 

Regarding Claim 6, Wang '979, Raz '154, and Jain '095 in combination disclose all the 
limitations of Claim 5 (supra). Further note that Wang '979, Raz '154, and Jain '095 in 
combination additionally disclose: when said data of the third type are not all equal to said lower 
predefined value Vmin, and are also not all equal to said higher predefined value Vmax, and are 
also not each equal either to Vmin or to Vmax, and also not all strictly between Vmin and Vmax (Raz 
' 1 54: Ins. 14-27 - note the hyperbox of Raz ' 154), said index of the example image consists of 
the result (X(Q)) of the product of said matrix (W) and said histogram of colors (Hm(Q)) 
representing the overall visual content of said example image (Q), and of said vector, (Gr(Q)) 
(Wang '979: col. 4, In. 62 to col. 5, In. 20 - see discusion in Claim 5 (supra)), each component 
of which consists of the histogram of colors representing the visual content of one of said regions 
of interest (Wang '979: col. 5, Ins. 4-5; col. 4, Ins. 28-32 - note the "segments" of Wang '979 
read on regions of interest). 
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Claims 7-9: 

Regarding Claims 7-9, Wang '979, Raz ' 154, and Jain '095 in combination disclose all 
the limitations of Claim 6 (supra). Further note that Wang '979, Raz '154, and Jain '095 in 
combination additionally disclose: 

- (Claim 7) said step of calculating a similarity between the example image and each of the 
images amongst at least one subset of the stored images, includes the step of calculating a 
similarity, denoted SIMi, obtained by means of the following formula: 

SIMj(D) = Max fH M (D) fl H M (ROI r S0 ) ] 

in which H M (D) designates a histogram of colors calculated for the stored image under 
consideration; ROI r S0 designates any region of interest in the example image for which 
the associated data item of the third type V r , is equal to Vmi n ; H M (ROI r S0 ) designates a 

histogram of colors calculated for this region of interest; the operator fl designates the 

intersection operation between histograms; and the function Max takes the largest value 
obtained by these intersections (Wang '979: col. 7, In. 24 to col. 8, In. 18). 
(Claim 8) said step of calculating a similarity between the example image and each of the 
images amongst at least one subset of the stored images, includes the step of calculating a 
similarity, denoted SIM2, obtained by means of the following formula: 

SIM 2 (D) = Max [ Hm(D) fl H M (ROI r SI ) ] 

in which H M (D) designates a histogram of colors calculated for the stored image under 
consideration; ROI r S0 designates any region of interest in the example image for which 
the associated data item of the third type V r , is equal to Vma X ; H M (ROI r sl ) designates a 
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histogram of colors calculated for this region of interest; the operator fl designates the 

intersection operation between histograms; and the function Max takes the largest value 
obtained by these intersections (Wang '979: col. 7, In. 24 to col. 8, In. 18). 
- (Claim 9) said step of calculating a similarity between the example image and each of the 
images amongst at least one subset of the stored images includes the step of calculating a 
similarity, denoted SIM 3 , obtained by means of the following formula: 



in which Hm(D) designates a histogram of colors calculated for the stored image under 
consideration; W designates said matrix; H M (Q) is a histogram of colors representing the 
global visual content of said example image (Q); and the operator n designates the 
intersection operation between histograms (Wang '979: col. 7, In. 24 to col. 8, In. 18). 
Note that the combination of Wang '979, Raz '154, and Jain '095 provide for use of appropriate 
similarity function as required by the user (Wang '979: col. 7, In. 32 and Jain '095: col. 8, Ins. 
20-55) and moreover provide for the elements to calculate the similarity function and conditions 
for the similarity functions of Claims 7-9. A person having ordinary skill in the art, faced with 
the problem of creating an appropriate similarity function as applicable per Claims 7-9 would 
reasonably expected to infer the similarity functions as per Claims 7-9 (see MPEP 2144.01). 



SIMs(D) =H M (D) r\X(Q) withX(Q) = W oH M (Q) 



Conclusion 

7. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 
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U.S. Patent No. 5,579,471 issued to Barber et al., "Image Query System and Method." Reference is the 
basic QBIC (query based on image content) patent. 

U.S. Patent No. 6,285,995 issued to Abdel-Mottaleb et al., "Image Retrieval System Using a Query Image." 
Reference provides another alternative base QBIC patent that uses color histograms. 
U.S. Patent No. 6,446,060 issued to Bergman et al., "System and Method for Sequential Processing for 
Content-Based Retrieval of Composite Objects." Reference teaches an alternative way to make a query 
from smaller subqueries. Subqueries read on individual regions of interest. 

U.S. Patent No. 5,963,670 issued to Lipson et al., "Method and Apparatus for Classifying and Identifying 
Images." Reference discloses the definition of a region of interest via 2-dim points (col. 2, In. 56 to col. 3, 
In. 18; col. 2, Ins. 9-16). 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Patrick J.D Santos whose telephone number is 703-305-0707. 
The examiner can normally be reached on M-F 8:00-4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Safet Metjahic can be reached on 703-308-1436. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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Patrick J.D. Santos 
May 13, 2004 
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